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What are more effective in English 
classrooms: textbooks or podcasts? 

Jaime Selwood 1 , Joe Lauer 2 , and Kazumichi Enokida 3 



Abstract. In the 21st century it has become clear that more and more language- 
learning pedagogical materials have begun to shift to a digital mobile-access format 
and away from being a textbook and classroom based one. High quality language- 
learning podcasts can provide a cheap, beneficial and portable technology that allows 
learners the freedom to access useful materials whenever and wherever convenient - 
especially through smartphones. However, despite the low cost involved in producing 
and accessing podcasts, educational institutions still seem reluctant to fully utilise 
them as an integral part of the language-learning process. This presentation will 
outline the structure of a comprehensive study that analysed the communicative 
abilities of 102 university freshmen students undertaking a compulsory English oral 
communication course. The study was divided into a traditional model that was 
centred-around a paper textbook and digital model where no textbooks were used 
and the course was structured around a series of high-quality, pedagogically sound 
English language podcasts. It was found that students progressed in similar ways 
during the course, regardless of the pedagogical materials used. 

Keywords: podcasts, mobile assisted language learning, authentic materials, 
smartphones. 


1. Introduction 

In a comprehensive study, the English communicative abilities of 102 university 
freshmen at a Japanese university were analysed in detail during one semester. 
Two instructors each taught two classes; the first being a traditional model which 
was structured around a textbook, whilst in the other course a digital model was 
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implemented which utilised high-quality, pedagogically strong English language- 
learning podcasts. Following Lauer, Selwood, and Enokida (2016), a series of 
communicative language-learning tests were administered at the beginning, middle 
and end of the semester to assess the student’s progress. The experimental model 
used in this study partially mirrored an influential study published by Gilmore 
(2011), which concluded that utilising authentic materials - which were mostly 
Internet-based - seemed to be more effective than traditional English-language 
learning textbooks. Thus, the goal of this study was to confirm Gilmore’s (2011) 
findings by determining which pedagogical materials were more effective in 
improving students’ oral abilities: textbooks or podcasts. 

This paper will outline how the study was implemented, state the benefits that can 
be found through using non-authentic language learning materials, and report on 
the results of the tests administered. 


2. Background 

The study attempted to determine if podcasts are more effective than textbooks 
in improving students’ English communicative abilities. Significantly, podcasts 
can be accessed through mobile digital devices such as smartphones or tablets. 
The potential of podcasts as a language-learning tool is that pedagogically 
strong podcasts can provide high-quality ‘authentic’ materials. As Bishop, 
Amankwatia, and Cates (2008) explain, “podcasts have the power to focus 
attention and assist language learners to acquire useful and authentic language” 
(pp. 467 - 486 ). 

English Language Teaching (ELT) textbooks, on the other-hand, often provide 
learners an impoverished or distorted view of the native language (Gilmore, 2007). 
The dialogues in textbooks are often centred-around contrived utterances, with 
the goal often being to illustrate a particular grammar point whilst highlighting 
frequent, easy to understood but unnatural vocabulary. 

Podcasts also have the potential to motivate because they are often published weekly 
or monthly and so students are provided with information as events unfold, whilst 
textbooks can often include language that has been specially adapted, thus losing 
its authenticity whilst often containing information and facts that soon become 
out-of-date. Another factor that is often undervalued is that most language-learning 
podcasts are free, so podcasts can offer a more egalitarian method to language- 
learners as the materials can be accessed free of charge. 
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The ultimate aim of a language-learning course should be to improve students’ 
language abilities and communicative competence. Traditionally in Japan, ELT 
textbooks are the go-to authentic material used on language-learning courses. Yet, 
in a prominent study, Gilmore (2011) concluded that students at a university in 
Japan significantly improved their communicative competence over a ten month 
period when they used web-based materials compared to a similar group of 
students who used only textbooks. The authors wanted to evaluate how successful 
podcasts could be when used as an essential tool in an English communicative 
course. By using Gilmore’s (2011) study as a starting point, they wished to contrast 
and evaluate the effectiveness of podcasts in comparison with textbooks. 


3. Method 

Two of the authors - Selwood and Lauer - each taught one podcast-centred 
course and one textbook-centred course. Each author chose different textbooks 
and podcasts to be used in the study. The podcasts selected by Selwood were 
English News Weekly, English News Students, BBC Six Minute Podcast and 
BBC The English We Speak. Lauer chose four different podcasts and these were 
Hiroshima University s English Podcast, VOA Podcast, Daily English Show and 
ECC Podcast. Selwood selected English Speak (Selwood, 2014) as his textbook 
whilst Lauer chose Slumdog Millionaire (Close, 2010). All the materials were 
selected because of their high pedagogical quality and their usage of authentic 
language. 

Each course lasted for one 15-week-semester and each class was 90 minutes 
in length and held once-a-week. Both the textbook and podcast courses were 
compidsory first year English communication classes. The students were assigned 
to each class depending on their faculty and their university entrance exam score 
and the TOEIC test (the standard English language test used in Japan - the highest 
score is 900 points). Lauer’s textbook class had an average score of 540/900 whilst 
his podcast class was 419/900. In contrast Selwood’s podcast class was 629/900 
whilst his textbook class was 467/900. This meant that each author had a ‘weaker’ 
and ‘stronger’ ability class and so the study could be measured amongst students 
of differing abilities. 

To assess students’ progress, three sets of tests were included in the survey - at the 
beginning, middle and end of the semester. The tests focused on grammar, listening 
and speaking because according to Canale and Swain (1980), these are the essential 
parts in measuring and analysing communicative competence. The three tests were: 


426 


What are more effective in English classrooms: textbooks or podcasts? 


• Communicative Vocabulary-Grammar Test. This measured students’ 
vocabulary and grammar skills. This test contained 49 questions, taken 
from a standard TOEIC test and the same 49 questions were used in all 
three tests, albeit with the question order changed from test to test. 

• Communicative Listening Test. This measured general listening ability 
through 2 1 questions taken from a TOEIC practice test. Again the question 
order was re-arranged for each test. 

• Three Minute Recorded Speaking Test. For this test students were 
placed in pairs and for three minutes the students entered into a dialogue 
recorded using smartphones. The audio was then emailed to their teacher 
for analysis. 


4. Results 

The Communicative Vocabulary-Grammar Test results showed only a tiny 
improvement in both courses. In the textbook classes the average score increased 
from 69% to 72% whilst the podcast courses showed a tiny increase from 69% 
to 70%. However, the stand-out result from the study was the improvement in 
Communicative Listening Test results in both courses. The textbook students 
improved their listening scores from an average, per student, from 43% to 54%. 
The podcast students also improved from an average score of 39% to 51% by the 
end of the semester. 

The Three Minute Recorded Speaking Test also showed small improvements in 
both courses. In the podcast course, each student, on average, increased their word 
count by 15 words per three minutes. However, in the podcast courses students 
only increased, on average, by seven words per three minutes. 


5. Conclusions 

The study found that, disappointingly, there were no differences in which authentic 
materials were used. The textbook courses worked just as well as the podcast ones. 
However, there were mitigating factors to why these results occurred. Firstly, the 
length of the study was only four months. Secondly, each class met just once a 
week for 90 minutes. Thirdly, the testing method used to assess the project was 
hindered by using the TOEIC test as this measures listening ability, vocabulary 
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knowledge and speaking skills. However the test is limited in how it measures 
students’ sociolinguistic abilities, discourse and strategic skills. The authors used 
this testing method as it had been used in Gilmore’s (2011) study as well as being 
the pre-eminent English language test used in Japanese universities. 

In the future, researchers wishing to investigate the merits of podcasts should 
focus on correcting the problems encountered by the authors on this study, namely 
expanding the length of time the study runs for. 
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